Skip to content

Add pre-session billing and cost notice - #436

Open
AchlysChang wants to merge 1 commit into
sysprog21:mainfrom
AchlysChang:pre-session-cost-warning
Open

AchlysChang wants to merge 1 commit into
sysprog21:mainfrom
AchlysChang:pre-session-cost-warning

Conversation

@AchlysChang

@AchlysChang AchlysChang commented Oct 22, 2026 •

Copy link
Copy Markdown

An interview runs on Gemini Live, which bills the Google AI project the interview runs on for the whole context of the session: retained audio and images are paid for again on every later turn (docs/provider-cost-and-degradation.md, "What an interview spends"). The final report, the quiet-pause interim reviews and the phase judge run on the same report model and keys, so they are billed to the same project. Neither the operator entering a key nor the candidate pressing Start was told so. This adds the pre-session notice #161 asks for; its session length cap and text-only mode stay open there.

Which project pays is the interview's, not always the server's: agent_config_for swaps in a LiveKit provider's own Google keys when it has any, so its rooms bill its project (docs/providers.md, "Gemini credential failover"). The wording holds for both. The Setup page says, in its Google AI section, that Gemini Live is billed to the project of the key an interview runs on, this key unless the interview's LiveKit provider sets Google keys of its own, and that the reports, interim reviews and phase judge run on the same keys and are billed to the same project.

The lobby shows a notice above Start, linked through aria-describedby, that says Gemini Live usage is billed on the whole context of the session to the Google AI project this interview runs on, and names what the chosen mode sends besides the text of the interview: voice and video for a coding interview with candidate video enabled, voice without it, and voice and board images at a whiteboard, where the camera is off but every board reaches Gemini as a realtime image on the same stream ("What a whiteboard interview adds"). In a coding interview with example drawing checked it also names the drawing, sent whenever Jim reads it: each read_board call sends the current example JPEG, which then stays in the billed context. Switching mode or toggling example drawing updates it.

The notice lists media against "the text of the interview" rather than claiming to be everything sent, because which text reaches the model depends on the mode, the loop and the candidate's consent: the code in a coding interview, which reaches the model as text and is resent on a reconnect, and, outside a coding-only session, any resume or job description snippets the candidate selected and agreed to send, built into the instructions of every generation. The doc paragraph in "What the candidate sees" spells that text out.

/api/session gains candidateVideo, answered by the room dispatcher: LocalDispatcher reports its agent config's setting, and any other dispatcher, or none, reports null. A web-only host cannot see the camera setting of an agent that joins from elsewhere, so it promises nothing, and the lobby keeps the notice hidden while the value is null, before the response arrives, and in offline practice when the request fails or a later refresh does.

The longest notice wraps to seven lines in the Start column, which put the 1280x800 lobby at 804px with taller font metrics, so the laptop media query sets the notice smaller and tighter. A lobby test of its own measures the page with that notice showing under the same three fonts and long title as the existing one-screen test, which is unchanged and runs with the notice hidden; the test fails at 804px without the rule.

Browser tests cover each mode, the switch between them, toggling and restoring example drawing, a null or missing value, a failed request and a failed refresh, and wait for the account status that /api/session sets rather than for the initial "Checking account..." text; the Setup page test checks both billing sentences. tests/web/accounts.rs checks candidateVideo as true, false and null in both the signed-out and the signed-in response, with null both from a dispatcher that does not know and from no dispatcher at all; a unit test covers LocalDispatcher reporting its config.

UI Verification

Setup page (Google AI section):
setup

Lobby — Coding mode (candidate video disabled / default):
lobby-coding-voice

Lobby — Coding mode (candidate video enabled):
lobby-coding-video

Lobby — Coding mode (candidate video enabled + example drawing checked):
lobby-coding-video-drawing

Lobby — Whiteboard mode:
lobby-whiteboard

jserv

This comment was marked as outdated.

cubic-dev-ai[bot]

This comment was marked as resolved.

@jserv jserv left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid ambiguous Refs #161. Always specify Close #? instead.

@AchlysChang
AchlysChang force-pushed the pre-session-cost-warning branch from 9d281e8 to 9c008bf Compare October 22, 2026 14:33
jserv

This comment was marked as outdated.

Comment thread web/app.js Outdated
Comment thread web/app.js Outdated
@AchlysChang
AchlysChang force-pushed the pre-session-cost-warning branch from 9c008bf to 9e6ab86 Compare October 23, 2026 03:21
Comment thread src/web/auth.rs Outdated
Comment thread web/app.js Outdated
Comment thread web/app.js Outdated
Comment thread web/app.js Outdated
Comment thread tests/browser/lobby.test.js Outdated

@jserv jserv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid ambiguous references such as Refs #161. Complete the implementation and close the issue properly.

@jserv jserv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI fails on the indent gate. Run make indent, then amend and force-push.

@jserv

jserv commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

@AchlysChang To settle the two requests about the issue reference: this PR covers the pre-session cost warning accepted on #161 on 2026-10-12, while the session length cap and the text-only toggle stay open there, so it does not close #161 and Refs #161 stays. The later request to close the issue is withdrawn. For Refs to stand on its own, CONTRIBUTING.md asks the body to state the motivation and the implementation in full: say where the notice appears (the setup page and the lobby), when it is shown or hidden, and which part of docs/provider-cost-and-degradation.md its wording follows. Also remove the leftover first line, "Draw problems from a saved practice list".

Comment thread tests/web/accounts.rs Outdated
@AchlysChang
AchlysChang force-pushed the pre-session-cost-warning branch 3 times, most recently from 2bfb083 to bd7460d Compare October 24, 2026 05:48
@AchlysChang
AchlysChang requested a review from jserv October 24, 2026 06:03

@jserv jserv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace Refs #161 in git commit messages with Close #. Don’t repeat the same mistakes.

Comment thread web/app.js Outdated
@AchlysChang
AchlysChang force-pushed the pre-session-cost-warning branch 2 times, most recently from e0ad75a to 1c0bfeb Compare October 24, 2026 06:59
@AchlysChang

Copy link
Copy Markdown
Author

Hi @jserv, regarding the commit message reference: yesterday you mentioned that since the session length cap and text-only toggle remain open, 'Refs #161 stays. The later request to close the issue is withdrawn.' (and also noted this in Issue #161).
Should I still change this to Close #161? If so, should I open a new separate issue to track the remaining session length cap feature?

@jserv

jserv commented Oct 24, 2026

Copy link
Copy Markdown
Contributor

Should I still change this to Close #161? If so, should I open a new separate issue to track the remaining session length cap feature?

Change to Close #161 in git commit messages.

@jserv

jserv commented Oct 24, 2026

Copy link
Copy Markdown
Contributor

Yesterday you mentioned that since the session length cap and text-only toggle remain open, 'Refs #161 stays. The later request to close the issue is withdrawn.' (and also noted this in Issue #161).

My instruction referred to the body of Git commit messages. You must fully resolve Issue #161 and include Close #161 at the end of the relevant commit message. The context matters.

@jserv jserv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before asking someone to review, you MUST resolve any regressions flagged by GitHub Actions.

@AchlysChang
AchlysChang force-pushed the pre-session-cost-warning branch from 1c0bfeb to aa4985d Compare October 24, 2026 08:21
Comment thread web/app.js Outdated
Comment thread src/web/setup.rs Outdated
jserv

This comment was marked as outdated.

@AchlysChang
AchlysChang force-pushed the pre-session-cost-warning branch from aa4985d to f750c38 Compare October 25, 2026 04:35
@jserv
jserv requested review from ColtenOuO and alanhc October 25, 2026 04:37
Comment thread web/app.js Outdated
An interview runs on Gemini Live, and its report, interim reviews and
phase judge run on the same keys, all billed to the Google AI project
the interview runs on; nothing told the operator or the candidate so
before it started. The Setup page now says so beside the key, including
that a LiveKit provider with Google keys of its own bills its rooms to
those, and the lobby names who pays and what reaches Gemini Live
besides the text of the interview: voice and video or voice, plus the
drawing whenever the interviewer reads it if example drawing is on, or
voice and board images at a whiteboard. /api/session reports
candidateVideo only when this process hosts the interviewer and null
otherwise, since the web half of a split deployment cannot see the
agent's settings, and the lobby shows the notice only when it knows.
This is the pre-session notice sysprog21#161 asks for; its session length cap
and text-only mode stay open there.
@AchlysChang
AchlysChang force-pushed the pre-session-cost-warning branch from f750c38 to 0cf5692 Compare October 25, 2026 12:31
Comment thread src/web/setup.rs
<fieldset>
<legend>Google AI <span class="requirement">Optional</span></legend>
<p class="group-note">Add a key if this server will host the interviewer.</p>
<p class="group-note">Gemini Live usage is billed on the whole context of the session, to the Google AI project of the key an interview runs on: this key, unless the interview's LiveKit provider sets Google keys of its own. Retained audio and images cost again on every later turn. The reports, interim reviews and phase judge run on the same keys as the interview and are billed to the same project.</p>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This holds only for a single key or keys in one project. ApiSurface::Live and ApiSurface::Report pick keys independently with separate cooldowns, and docs/providers.md says report failover leaves the Live key alone. With GOOGLE_API_KEYS spanning two projects, a report can rotate to the second key while Live stays on the first, so it bills a different project. Say that they draw from the same key list and each request is billed to the project owning the key it used, and update the matching assertion in setup-page.test.js.

Jim reads it: each `read_board` call sends the current example JPEG, which then
stays in the context like any other image. The Setup page says the same beside
the Google API key, naming that key's project unless the interview's LiveKit
provider sets Google keys of its own, and that the final report, the quiet-pause

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same overstatement as the Setup text: "run on the same report model and keys as the interview, so they are billed to the same project" is wrong once the key list spans projects, because the Report surface fails over on its own. Reword it to match the corrected Setup text.

Comment thread web/app.js
} catch {
// A server that cannot answer about accounts is not one that will mint a
// token either, but the editor still works offline, so do not lock the page.
// Nor will it bill one, so a notice from an earlier answer comes down.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is not true: a failed /api/session leaves Start enabled, and the interview can still navigate and get a token from a server that bills Gemini. The notice then disappears when it is still needed. If hiding it on failure is the policy, say the reason is that we cannot tell who pays, not that nothing will be billed.

@jserv jserv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebase onto the latest main branch, fix any regressions detected by GitHub Actions, and address the review comments.

Comment thread web/styles.css
Comment on lines +3891 to 3894
font-size: 0.8125rem;
line-height: 1.35;
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test the longest billing notice keeps the lobby on one 1280x800 screen still fails at this head in the browser gate: with the system font the page is 808px tall against an 800px viewport, and the notice is 122px high, so this rule does not buy the room the comment above it claims. Shorten what the longest notice says or tighten the Start column further, and confirm against the CI system font rather than only the overridden metrics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants